A Central Limit Theorem for Temporally Nonhomogenous Markov Chains with Applications to Dynamic Programming

Stochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry

We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...

Central and Local Limit Theories in Markov Dependent Random Variables

Moderate Deviations for Time-varying Dynamic Systems Driven by Nonhomogeneous Markov Chains with Two-time Scales∗

Motivated by problems arising in time-dependent queues and dynamic systems with random environment, this work develops moderate deviations principles for dynamic systems driven by a fast-varying nonhomogeneous Markov chain in continuous time. A distinct feature is that the Markov chain is time dependent or inhomogeneous so are the dynamic systems. Under irreducibility of the nonhomogeneous Mark...

Central Limit Theorem for Hitting times of Functionals of Markov Jump Processes

A sample of i.i.d. continuous time Markov chains being defined, the sum over each component of a real function of the state is considered. For this functional, a central limit theorem for the first hitting time of a prescribed level is proved. The result extends the classical central limit theorem for order statistics. Various reliability models are presented as examples of applications. Mathem...

The Central Limit Theorem for the Normalized Sums of Extended Sliding Block Codes from Sequences of Markov Chains with Time Delay

We extend the sliding block code in symbolic dynamics to transform two sequences of Markov chains with time delay. Under the assumption that chains are irreducible and aperiodic, we prove the central limit theorem (CLT) for the normalized sums of extended sliding block codes from two sequences of Markov chains. We apply the theorem to evaluations of bit error probabilities in asynchronous sprea...